Bayesian retrieval in associative memories with storage errors
نویسندگان
چکیده
It is well known that for finite-sized networks, onestep retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have hitherto only had heuristic justification. We show how they emerge naturally from considerations of probabilistic inference under conditions of noisy and partial input and a corrupted weight matrix. We start from the conditional probability distribution over possible patterns for retrieval. This contains all possible information that is available to an observer of the network and the initial input. Since this distribution is over exponentially many patterns, we use it to develop two approximate, but tractable, iterative retrieval methods. One performs maximum likelihood inference to find the single most likely pattern, using the (negative log of the) conditional probability as a Lyapunov function for retrieval. In physics terms, if storage errors are present, then the modified iterative update equations contain an additional antiferromagnetic interaction term and site dependent threshold values. The second method makes a mean field assumption to optimize a tractable estimate of the full conditional probability distribution. This leads to iterative mean field equations which can be interpreted in terms of a network of neurons with sigmoidal responses but with the same interactions and thresholds as in the maximum likelihood update equations. In the absence of storage errors, both models become very similiar to the Willshaw model, where standard retrieval is iterated using a particular form of linear threshold strategy.
منابع مشابه
Modeling storage and retrieval of memories in the brain
We have proposed a neural network model that stores the incoming information after orthogonalizing it in the same manner as vectors are orthogonalized. The scheme enables the brain to compare a new informational system with those in the memory and store its similarities and differences with the old memories in an economical manner. This allows the brain to have an enormous capacity and yet the ...
متن کاملModeling storage and retrieval of memories in the brain
We have proposed a neural network model that stores the incoming information after orthogonalizing it in the same manner as vectors are orthogonalized. The scheme enables the brain to compare a new informational system with those in the memory and store its similarities and differences with the old memories in an economical manner. This allows the brain to have an enormous capacity and yet the ...
متن کاملRobust image retrieval from noisy inputs using lattice associative memories
Lattice associative memories also known as morphological associative memories are fully connected feedforward neural networks with no hidden layers, whose computation at each node is carried out with lattice algebra operations. These networks are a relatively recent development in the field of associative memories that has proven to be an alternative way to work with sets of pattern pairs for w...
متن کاملNeurogenesis and multiple plasticity mechanisms enhance associative memory retrieval in a spiking network model of the hippocampus
Hippocampal CA3 is crucial for the formation of long-term associative memory. It has a heavily recurrent connectivity, and memories are thought to be stored as memory engrams in the CA3. However, despite its importance for memory storage and retrieval, spiking network models of the CA3 to date are relatively small-scale, and exist as only proof-of-concept models. Specifically, how neurogenesis ...
متن کاملEnhanced storage capacity with errors in scale-free Hopfield neural networks: An analytical study
The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of O(N), where N is the system size. Beyond the threshold, they are completely lost. Since the introduction of the Hopfield model, the theory of neural network...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- IEEE transactions on neural networks
دوره 9 4 شماره
صفحات -
تاریخ انتشار 1998